Add Python task execution options authoring (v0.1.10) - #52

Problem

Python-authored pipelines cannot emit task-level executionOptions, so there is no way to author a non-cached task. TaskNode has no field for it and _emit_task writes only annotations / componentRef / arguments / isEnabled. Tangle's "do not cache this task" knob — executionOptions.cachingStrategy.maxCacheStaleness: P0D — is unreachable from Python.

This blocks tasks that read runtime state (e.g. a run's CLOUD_PIPELINES_PIPELINE_RUN_CREATED_BY), where a cached result is silently wrong.

Authoring API

Two reserved call-site keywords on container-component tasks, following the existing is_enabled= precedent:

# narrow ergonomic knob for the common no-cache case
gate = read_runtime_state(name="...", max_cache_staleness="P0D")

# general passthrough for the rest of ExecutionOptionsSpec
uploaded = flaky_upload(
    payload=data.Output,
    execution_options={"retryStrategy": {"maxRetries": 3}},
)

Emits:

executionOptions:
  cachingStrategy:
    maxCacheStaleness: P0D

• max_cache_staleness= wins over a cachingStrategy.maxCacheStaleness supplied through execution_options=, preserving sibling fields. The authored mapping is deep-copied, so a shared constant reused across tasks is never mutated.
• Works on ref(...), @task and @registered tasks (one shared call path).
• Same collision escape as is_enabled: a component input of that name is bound with .bind(...) and stays under arguments.
• Subpipeline (graph-component) tasks reject both keywords, mirroring is_enabled — caching and retries apply to the container executions inside the child graph.
• Emitted after isEnabled in canonical task key order.

Schema corrections

Reviewers questioned the timeout / retryStrategy.backoff fields used in the first draft of the docs. They do not exist. Verified against the pinned backend (cloud_pipelines_backend.component_structures, ExecutionOptionsSpec = caching_strategy + retry_strategy; RetryStrategySpec = required max_retries), and reproduced through the model itself:

ExecutionOptionsSpec.from_json_dict(
    {"retryStrategy": {"maxRetries": 3, "backoff": "30s"}, "timeout": "30m"}
).to_json_dict()
# -> {"retryStrategy": {"maxRetries": 3}}

The backend's models leave extra unset (extra="ignore"), so those keys are silently discarded on submit — the worst failure mode, since docs would promise a timeout that never takes effect. dehydrated_pipeline_schema.json was the only place they existed; the generated pipeline_schema.json and the OpenAPI schema never had them.

• Dropped executionOptions.timeout and retryStrategy.backoff from the dehydrated schema.
• Added required: ["maxRetries"] to RetryStrategySpec, matching the generated schema and the backend's non-Optional field. Previously {"retryStrategy": {}} passed dehydrated validation and then failed server-side.

Because unmodeled keys are ignored rather than rejected, the execution_options= passthrough validates against the fields the backend actually models rather than accepting anything — consistent with the fail-closed treatment of every other authored value. test_execution_option_fields_match_generated_schema pins that allowlist to pipeline_schema.json (regenerated from the backend by scripts/refresh_pipeline_schema.py), so a backend field addition fails a test instead of silently staying unauthorable.

Review history

Reviewed locally before submission; all findings resolved:

1. Advertised-but-nonexistent timeout/backoff (example, README, docstrings, subpipeline error text) — removed; schema aligned rather than cementing the divergence.
2. Compile accepted a server-invalid empty retry strategy — required: ["maxRetries"] added at the schema layer plus an actionable emit-time error; regression cases added.
3. A malformed cachingStrategy was masked by the narrow-wins merge ({"cachingStrategy": "bad"} silently replaced) — validation now runs before the merge, so scalar/null parents raise instead of being overwritten.

Note on the schema edit: removing the ghost properties does loosen validation for those keys (timeout: 3 was previously rejected, now accepted under additionalProperties: true). That is deliberate — constraining the shape of fields the backend discards enforces nothing, and compiler-authored input is fail-closed via the allowlist. The maxRetries requirement is the intentional tightening.

Version

Patch bump 0.1.9 → 0.1.10 across pyproject.toml, uv.lock, tangle_cli.__init__ fallback, and the packaging test assertion.

Tests

1126 passed locally (up from 1108), including the packaging suite building a wheel at 0.1.10. Coverage added for: emitted shape and key order, narrow-wins merge, no mutation of shared mappings, @task path, unknown/partial/malformed option rejection, subpipeline rejection for both keywords, dehydrator round-trip, schema accept/reject cases, and the allowlist-vs-generated-schema drift guard.

End-to-end verification: the example compiles, and the emitted executionOptions round-trip losslessly through the real backend model for both tasks.